Various machines, such as those that are used to dig, loosen, carry, compact, etc., different materials, may be equipped with object detection and recognition systems that incorporate devices such as radio detection and ranging (radar) devices and/or cameras. In some applications, machines use object detection and recognition devices for safety. For example, in one application, autonomous or semi-autonomous machines may use object detection devices to detect objects in areas surrounding the machines as part of a collision avoidance mechanism. In another application, object detection devices can assist an operator of large machines by detecting objects that are out of the operator's field of view, classifying the objects, and initiating a safety protocol based on the classification of the object.
Some object detection and recognition systems are radar based and only use radar data because radar data can be processed quickly. One downside to radar based object detection and recognition systems, however, is that they offer unsatisfying performance as radar data lacks the specificity needed to accurately distinguish between two objects of different class (for example, a person or a light vehicle). On the other hand, object detection and recognition systems relying on image data from cameras must constantly process large amounts of data in real-time, or near real-time, using complex algorithms. For example, when a large machine is equipped with multiple cameras covering all sides of the large machine, the object detection and recognition system may constantly receive streams of data from all of the and cameras and process it using computationally expensive image processing techniques. Accordingly, an object detection and recognition system that offers the speed of radar based systems and offers the accuracy of image based systems may be desirable, especially in applications involving large machines.
One system attempting to decrease the computation time needed to process object detection in images is disclosed in U.S. Pat. No. 8,068,134 to Yoshizawa, which issued on Nov. 29, 2011 (the '134 patent). The system in the '134 patent incorporates a radar sensor scanning the front of a passenger vehicle and a camera taking images in front of the passenger vehicle. The radar sensor detects objects within a range and direction in front of the vehicle, and a portion of an image corresponding to the range and direction is processed to detect an object.
Although the system of the '134 patent may be suitable for passenger vehicles, it may not be suitable for applications involving large machines. For example, object detection and recognition systems of large machines generally require object detection on all sides of the vehicle as an operator's field of view may be limited on all sides of the vehicle. Further, many object detection and recognition systems of large machines classify objects according their type (for example, equipment, large vehicle, or person) which requires additional processing and computing resources. As a result, large vehicle object detection and recognition systems must process more data faster than what could be accomplished using the system disclosed in the '134 patent. Accordingly, additional performance and accuracy advantages beyond the system of the '134 patent may be desirable.
The disclosed object recognition system is directed to overcoming one or more of the problems set forth above and/or other problems of the prior art.